While buying a house, the most important questions a buyer may ask himself are - ’What is the price of the house?’ and ’With such price, what features I may get?’.

Based on these queries it is relevant to look for the features which are affecting the house prices and to observe how the features are affecting the prices.

Here our primary objective is to build a linear regression model for predicting House Price.

The Ames Housing dataset which consists of 79 different explanatory variables. It is enriched with data on almost every aspect of a house. These explanatory housing parameters focus on the quality and quantity of many physical attributes of the property.

Total number of observations: 1459

There are certain categorical predictors having too many levels (eg. ‘Neighborhood’ has 27 levels) as well as predictors having only 2 levels.

• Quantitative: 1stFlrSF, 2ndFlrSF, 3SsnPorch, BedroomAbvGr, BsmtFinSF1, BsmtFinSF2, BsmtFullBath, BsmtHalfBath, BsmtUnfSF, EnclosedPorch, Fireplaces, FullBath, GarageArea, GarageCars, GarageYrBlt, GrLivArea, HalfBath, KitchenAbvGr, LotArea, LotFrontage, LowQualFinSF, MSSubClass, MasVnrArea, MiscVal, MoSold, OpenPorchSF, OverallCond, OverallQual, PoolArea, ScreenPorch, TotRmsAbvGrd, TotalBsmtSF, WoodDeckSF, YearBuilt, YearRemodAdd, YrSold

• Qualitative: Alley, BldgType, BsmtCond, BsmtExposure, BsmtFinType1, BsmtFinType2, BsmtQual, CentralAir, Condition1, Condition2, Electrical, ExterCond, ExterQual, Exterior1st, Exterior2nd, Fence, FireplaceQu, Foundation, Functional, GarageCond, GarageFinish, GarageQual, GarageType, Heating, HeatingQC, HouseStyle, KitchenQual, LandContour, LandSlope, LotConfig, LotShape, MSZoning, MasVnrType, MiscFeature, Neighborhood, PavedDrive, PoolQC, RoofMatl, RoofStyle, SaleCondition, SaleType, Street, Utilities.

Here our response is Price ( ‘SalePrice’).It is a continuous variable and unit is US Dollars.

A brief summary of the response is given below:

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  135751  168703  179209  179184  186789  281644

<>

The dataset originally contains 79 predictors.

We drop the following columns:

We introduce new predicting column: ‘Age_Sold_Group’, ‘renov_bi’

• ‘Age_Sold_Group’ = ‘YrSold’-‘YearBuilt’
• ‘renov_bi’ can take two values. 1 if the house was renovated or 0 if the house was not renovated.

Then, we create dummy of all the categorical predictors.

As we saw that the response is highly positively skewed, we may apply a log transformation over the response.

Histogram of response after log transformation:

We split the dataset into two parts. 80% of the whole data is contained in train set and the remaining 20% is contained in the test set.

Here we use Least Square Linear Regression Model.

Response: log(‘SalePrice’)

Since the dataset contains large number of predictors and also since multicollinearity is present in the dataset, it is wise to remove unnecessary predicting features which do not have significant impact on the response.

Recall that from the correlation heatmap, we noticed price is highly correlated with ‘LotArea’, ‘BedroomAbvGr’, ‘TotalBsmtSF’.

##                  Estimate   Std. Error    t value Pr(>|t|)
## (Intercept)  1.177705e+01 2.312430e-03 5092.93330        0
## LotArea      1.017735e-05 1.216701e-07   83.64706        0
## BedroomAbvGr 7.538112e-02 7.301208e-04  103.24472        0

• Multiple R-Squared: 0.9475
• Residual Standard Error: 0.0204400
• PRESS:0.4926025

## 791 729 
## 623 575

Here we compare two variable selection method:

• LASSO

• Principle Component Analysis

From LASSO, we get the following predicting columns:

##  [1] "(Intercept)"          "LotFrontage"          "LotArea"             
##  [4] "YearRemodAdd"         "HalfBath"             "BedroomAbvGr"        
##  [7] "KitchenAbvGr"         "TotRmsAbvGrd"         "GarageCars"          
## [10] "EnclosedPorch"        "MSSubClass_30"        "MSSubClass_60"       
## [13] "MSSubClass_80"        "MSSubClass_90"        "MSSubClass_120"      
## [16] "MSSubClass_180"       "MSZoning_FV"          "MSZoning_RL"         
## [19] "Alley_Pave"           "LotShape_IR2"         "LotShape_IR3"        
## [22] "LotConfig_Inside"     "LandSlope_Mod"        "LandSlope_Sev"       
## [25] "Neighborhood_Blueste" "Neighborhood_BrkSide" "Neighborhood_CollgCr"
## [28] "Neighborhood_Crawfor" "Neighborhood_SWISU"   "Condition1_PosN"     
## [31] "Condition1_RRAe"      "Condition1_RRAn"      "BldgType_Duplex"     
## [34] "BldgType_Twnhs"       "BldgType_TwnhsE"      "HouseStyle_2.5Unf"   
## [37] "OverallQual_2"        "OverallCond_4"        "OverallCond_6"       
## [40] "OverallCond_7"        "Exterior2nd_Brk Cmn"  "Exterior2nd_BrkFace" 
## [43] "Exterior2nd_Plywood"  "Exterior2nd_Wd Shng"  "MasVnrType_BrkFace"  
## [46] "MasVnrType_Stone"     "Foundation_CBlock"    "Heating_Grav"        
## [49] "HeatingQC_Gd"         "HeatingQC_Po"         "Functional_Min1"     
## [52] "FireplaceQu_TA"       "GarageType_Attchd"    "GarageQual_Gd"       
## [55] "GarageCond_Po"        "PavedDrive_P"         "MoSold_2"            
## [58] "MoSold_3"             "MoSold_4"             "MoSold_5"            
## [61] "MoSold_6"             "MoSold_7"             "MoSold_8"            
## [64] "MoSold_9"             "MoSold_10"            "MoSold_11"           
## [67] "MoSold_12"            "SaleType_CWD"         "SaleType_New"        
## [70] "SaleType_Oth"

We use the predicting columns obtained from LASSO and fit a least square linear model.

We train our model using the training set and obtain the following results:

• Multiple R-squared: 0.9585
• Residual Standard Error: 0.0186
• PRESS: 2833.417

One way to reduce the number of predictor variables is Principal Component Analysis(PCA). Here, we transform the numerical predictors into orthogonal set of predicting variables such that the new predicting variables explain 95% of the total variation.

Here, through PCA we obtain 21 principal components that explains 95% of the total variation.

We use the 21 principal components and all other categorical features and fit Least Square Linear Model over the response variable.

We train our model using the training set and obtain the following results:

• Multiple R-squared: 0.97210
• Residual Standard Error: 0.01698
• PRESS: 0.50811

So, we can say that predictors chosen by LASSO gives a better fit to the response variable.